1. Field of the Invention
Embodiments presented in this disclosure generally relate to generating depth maps, and more specifically, to generating depth maps by fusing data received from sensors of different modalities.
2. Description of the Related Art
A computing device may be coupled to a camera or a combination of sensors to assist in producing higher quality images or in reconstructing a scene. One way of accomplishing this is by performing depth sensing where the distance between a point in a scene and a camera (usually a reference camera) is determined. By measuring the depth for a plurality of pixels in a reference image, a depth map for the reference image may be acquired.
To measure depth, some systems use a time of flight (ToF) sensor. ToF sensors send out light and measure the time it takes for the light to travel from the sensor to an object, and then back to the sensor. By measuring this round-trip travel time, the distance from a plurality of points in the scene to the ToF sensor may be measured in real time. For example, the ToF sensor may be a scannerless light detection and ranging (LIDAR) sensor which generates a plurality of depth measurements for a plurality of different points that may be used to form a depth map for an image. However, ToF sensors, especially inexpensive ones, may not provide a depth measurements at a resolution level sufficient to support the depth map computation of a reference image (e.g., an HD image), resulting in low quality depth maps which may limit the types of processing that can be performed on the reference image. For example, a pixel-accurate depth map may increase the ability to distinguish between different layers of the reference image when adding special effects.